Dynamic sequencing of data partitions for optimizing memory utilization and performance of neural networks

ABSTRACT

Optimized memory usage and management is crucial to the overall performance of a neural network (NN) or deep neural network (DNN) computing environment. Using various characteristics of the input data dimension, an apportionment sequence is calculated for the input data to be processed by the NN or DNN that optimizes the efficient use of the local and external memory components. The apportionment sequence can describe how to parcel the input data (and its associated processing parameters—e.g., processing weights) into one or more portions as well as how such portions of input data (and its associated processing parameters) are passed between the local memory, external memory, and processing unit components of the NN or DNN. Additionally, the apportionment sequence can include instructions to store generated output data in the local and/or external memory components so as to optimize the efficient use of the local and/or external memory components.

RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/486,432, filed on Apr. 17, 2017 and titled “Enhanced NeuralNetwork Designs,” the entire disclosure of which is incorporated in itsentirety by reference herein.

BACKGROUND

Neural network (NN) or deep neural network (DNN) processing typicallyrequires that runtime occurs over multiple layers of the network whereactivations are computed at each layer. The output of the first layeracts as the input of the subsequent layer. For example, the processedactivations of a first NN/DNN layer act as inputs to a second NN/DNNlayer. The processed activations of a second layer then act as input toa third layer. The processed activations continue to occur untilreaching the last layer of the network where the output of the lastlayer is used by the NN/DNN computing environment for presentation orstorage.

Currently deployed NN/DNN computing environments, having limitedresources such as local memory (i.e. local caches), typically storeintermediate layer activations to a main memory. Operatively, data beingprocessed is transferred from the local memory/processing unit(s) to themain memory and then back to the local memory/processing unit(s), i.e.,when the data is needed for processing the next layer. Such a practiceis generally inefficient and requires undergoing avoidable processingcycles as well as the use of critical memory management resources,contributing to latency and stressed performance of the NN/DNN computingenvironment.

Moreover, NNN/DNN computing environments that use a small local memoryas a staging area for its inputs/activations and weights can beinefficient since all intermediate activations are generally stored tothe main memory. For example, if the size of the inputs and weights fora given layer is larger than the size of the local memory, the outputsof the current layer cannot be stored to the local memory withoutoverwriting the inputs and weights being processed. Operatively, theoutputs of this given layer will need to be stored to the main memory.If the next layer in the network is a layer that consumes this data (asmost of the time is the case), then the system is utilizing double thebandwidth by storing the data to the main memory and then copying itback to the local staging memory for required processing by the nextlayer of the network.

A more advantageous NN/DNN architecture/data management scheme wouldmaximize the use of local data (i.e., local to the processing unit(s))and minimize data read/writes to the main memory, which would result ina net benefit in terms of processing speed and power consumption.

It is with respect to these considerations and others that thedisclosure made herein is presented.

SUMMARY

Techniques described herein provide for the use of a “depth first”and/or a dynamic “depth first” approach to data processing utilized inan exemplary neural network (NN) and/or Deep Neural Network (DNN)environment, wherein the “depth first” and/or “dynamic depth first”processing protocol (e.g., expressed as one or more instructionsprovided by a controller component of the exemplary NN and/or DNNenvironment) operatively calculates and executes a data apportionmentsequence that allows for the processing of data that improves overallperformance and optimizes memory management. In other illustrativeimplementations, the data apportionment sequence can be calculated byother cooperative components of the exemplary neural network (NN) and/orDeep Neural Network (DNN) environment including but not limited toonline or offline compilers and other associated components.

In an illustrative implementation, an exemplary DNN environment cancomprise one or more processing blocks (e.g., computer processingunits—CPUs), a memory controller, a high bandwidth fabric (e.g., databus passing data and/or data elements between an exemplary DNN moduleand the cooperating components of a DNN environment), an operationcontroller, and a DNN module. In the illustrative implementation, theexemplary DNN module can comprise an exemplary DNN state controller, adescriptor list controller (DLC), dMA (DDMA), DMA Streaming Activations(DSA), an operation controller, a load controller, and a storecontroller.

In an illustrative operation, the operational controller of the NN/DNNenvironment can operatively process large amounts of data in order toapply one or more desired data processing operations (e.g., convolution,max pooling, scalar multiply/add, summation, fully connected, etc.). Inthe illustrative operation, a participating user can specify thedimensions of the data being processed by the NN/DNN environment.Illustratively, using the data dimensions, the number of availableprocessing layers in the NN/DNN environment, as well as datarepresentative of one or more characteristics of cooperating memorycomponents of the NN/DNN environment (e.g., memory size, location,latency, efficiency, etc.), a data apportionment sequence can becalculated by the NN/DNN environment components that specifies thatinput data for each layer is to be apportioned (as well as anyassociated processing parameters) and communicated between thecooperating NN/DNN memory components and NN/DNN processors to achieveoptimal processing.

In an illustrative implementation, the exemplary NN/DD environment caninclude a local memory component and an external memory component. Inthis implementation, the local memory components operatively transferdata at higher rates with reduced latency relative to the externalmemory component. Operatively, the local memory component can include amemory size to store smaller amounts of data relative to the externalmemory component.

In an illustrative operation, input data (e.g., a data blob) can bereceived for processing by the NN/DNN environment having a specificdefined data dimension, associated data processing parameters (e.g.,layer weights), a defined number of processing layers required forprocessing, as well as data representative of one or morecharacteristics of the cooperating memory components of the NN/DNNenvironment. Operatively, the operational controller providesinstructions to calculate a data apportionment sequence using thereceived data dimensions, cooperating memory characteristics, and numberof layers. The calculated data apportionment sequence generates datarepresentative of the number of portions to parcel the input data acrosseach of the available processing layers and the timing of loading of theportions of data (and their associated processing parameters) from anexternal memory component, to an internal memory component, to anavailable processing unit of the NN/DNN environment.

Additionally, the calculated data apportionment sequence can includeinstructions for the operational controller to communicate the dataportion(s) from the external memory to the local memory to the availableprocessing unit(s) according to a calculated sequence. Illustratively,the local memory component can be utilized in the processing sequence tostore output data for processed portions of data for each of theprocessing layers. In the illustrative operation, upon processing all ofthe portions of a given processing layer and storing such output portiondata in a local memory, the cooperating processing units can assemblethe generated output portion data for the given processing layer (storedin the local memory) to generate complete output data for the givenprocessing layer. In an illustrative operation, the generated completelayer output data can then be stored in the external memory componentthat results in making more of the local memory component's memoryavailable for subsequent layer processing.

The techniques presented herein provide advantageous NN/DNNarchitecture/data management schemes that can maximize the use of localdata (i.e., local to the processing unit(s)) and minimize dataread/writes to the main memory, which result in a net benefit in termsof processing speed and power consumption.

It should be appreciated that, although described in relation to asystem, the above-described subject matter may also be implemented as acomputer-controlled apparatus, a computer process, a computing system,or as an article of manufacture such as a computer-readable mediumand/or dedicated chipset. These and various other features will beapparent from a reading of the following Detailed Description and areview of the associated drawings. This Summary is provided to introducea selection of concepts in a simplified form that are further describedbelow in the Detailed Description.

This Summary is not intended to identify key features or essentialfeatures of the claimed subject matter, nor is it intended that thisSummary be used to limit the scope of the claimed subject matter.Furthermore, the claimed subject matter is not limited toimplementations that solve any or all disadvantages noted in any part ofthis disclosure.

DRAWINGS

The Detailed Description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Thesame reference numbers in different figures indicate similar oridentical items. References made to individual items of a plurality ofitems can use a reference number with a letter of a sequence of lettersto refer to each individual item. Generic references to the items mayuse the specific reference number without the sequence of letters.

FIG. 1 illustrates a block diagram of an exemplary neural networkenvironment in accordance with the herein described systems and methods.

FIG. 2 illustrates a block diagram of an exemplary neural networkenvironment deploying the data apportionment sequence in accordance withthe herein described systems and methods.

FIG. 3 illustrates a block diagram of exemplary input data representedin an illustrative logical data mapping according to the hereindescribed systems and methods.

FIG. 4 illustrates a block diagram of an exemplary block sequencediagram for exemplary input data processed according to an illustrativeapportionment sequence in accordance with the herein described systemsand methods.

FIG. 5A illustrates a block sequence diagram of an exemplary stepwiseprocessing sequence for exemplary input data processed according toanother illustrative apportionment sequence in accordance with theherein described systems and methods.

FIG. 5B illustrates exemplary processing sequences for variousprocessing sequences in accordance with the herein described systems andmethods.

FIG. 6 is a flow diagram of an illustrative process to allow for a depthfirst processing of data in an exemplary neural network environment.

FIG. 7 is a flow diagram of an illustrative depth first processing oflayers of an exemplary neural network environment.

FIG. 8 shows additional details of an illustrative computer architecturefor a computer capable of executing the herein described methods.

FIG. 9 shows additional details of illustrative computing devicescooperating in accordance with the herein described systems and methods.

DETAILED DESCRIPTION

The following Detailed Description describes techniques that provide forthe optimization of processing and memory resources utilized in anexemplary neural network (NN) and/or Deep Neural Network (DNN)environment. In general, the iterators (e.g., expressed as iteratorcontroller components of the exemplary NN and/or DNN environment)operatively allow for the processing of data that improves overallperformance and optimizes memory management. In an illustrativeimplementation, an exemplary DNN environment can comprise one or moreprocessing blocks (e.g., computer processing units—CPUs), a memorycontroller, a high bandwidth fabric (e.g., data bus passing data and/ordata elements between an exemplary DNN module and the cooperatingcomponents of a DNN environment), iterator controller, operationcontroller, and a DNN module. In the illustrative implementation, theexemplary DNN module can comprise an exemplary DNN state controller, adescriptor list controller (DLC), dMA (DDMA), DMA Streaming Activations(DSA), operation controller, load controller, and store controller.

In an illustrative operation, the operational controller of the NN/DNNenvironment can operatively process large amounts of data in order toapply one or more desired data processing operations (e.g., convolution,max pooling, scalar multiply add, summation, fully connected, etc.). Inthe illustrative operation, a participating user can specify thedimensions of the data being processed as well as the configuration dataprocessing by the NN/DNN environment. Such data can include the numberof processing layers available for processing data in the NN/DNNenvironment as well as one or more operational characteristics of thecooperating memory components of the NN/DNN environment.

In an illustrative implementation, data to be processed by the NN/DNNenvironment can be represented as a blob. Generally, a blob representsthe data in memory that needs to be processed. Each blob can maintain alogical mapped shape defined by various dimensions such as width,height, number of channels, number of kernels, and other availabledimensional units. In an illustrative operation, the components of theNN/DNN can traverse across a multi-dimensional blob (e.g., as defined bya logical data mapping) or a smaller N dimensional slice of such a blob,where N is the number of dimensions (e.g., for a 3D blob representing animage with width, height and number of channels—N=3). While traversingthe blob, one or more instructions can be generated by one or morecooperating components of the NN/DNN including, but not limited to, loadinstructions for loading data from the source memory to the processingunit(s) (e.g., neuron processors), or store instructions for storingdata produced by the processing unit(s) to a destination memory (e.g.,cooperating memory component of the NN/DNN environment—local or externalmemory). In the illustrative operation, the operational controller iscapable of producing instructions that read/write multiple dataconcurrently.

Generally, in processing data in a Neural Network (NN), there can bemultiple processing layers at runtime wherein the activations at eachlayer can be computed and then communicated as an input to a subsequentlayer, until reaching the last layer of the network. When implementingsystems with limited resources such as local memory (e.g., localcaches), there can exist a need to store the intermediate layeractivations to a cooperating larger memory component (e.g., main memory,external memory). Currently, such data transfers consume avoidableprocessing as data transfers from the local memory/processing unit(s) tothe main memory and then back to the local memory/processing unit(s)will occur when the data is needed for processing by the next layer.

The herein described systems and methods provide a mechanism throughwhich the use of local data (i.e., data local to the processing unit(s)and stored on a local memory component) is optimized and the datatransfers that are required to occur to/from the main memory areminimized. This can result in a net benefit in terms of increasingprocessing speed and reducing power consumption.

Typically, NN/DNN are operative to perform the processing execution ofeach complete layer before processing the next layer in the network.Moreover, typically, intermediate layer activations (i.e., intermediatelayer generated output data) are generally stored in a main memory untilconsumed by the next processing layer.

In an illustrative implementation, an apportionment sequence can becalculated that parcels the input data into processing portions acrossthe available processing layers that includes instructions to storeintermediate activations in a cooperating local memory component andstore complete output data in a cooperating main memory component. Theapportionment sequence can be calculated using the input datadimensions, data representative of the operational characteristics ofthe cooperating memory components (e.g., size, speed, location of thememory components, etc.), number of neuron processors available forprocessing data, clock speed of the processing units, as well as thenumber of available processing layers in the NN/DD environment.Illustratively, the calculated apportionment sequence provides aspecific sequence of which portions of data (and associated processingparameters—e.g., layer weight data) to process on a step by step basis,as well as where to load and/or store the data (i.e., local memorycomponent or main memory component) on a step by step basis.

Techniques described herein provide for the use of a “depth first”and/or a dynamic “depth first” approach to data processing utilized inan exemplary neural network (NN) and/or Deep Neural Network (DNN)environment, wherein the “depth first” and/or “dynamic depth first”processing protocol (e.g., expressed as one or more instructionsprovided by a controller component of the exemplary NN and/or DNNenvironment) operatively calculates and executes a data apportionmentsequence that allows for the processing of data that improves overallperformance and optimizes memory management. In other illustrativeimplementations, the data apportionment sequence can be calculated byother cooperative components of the exemplary neural network (NN) and/orDeep Neural Network (DNN) environment including but not limited toonline or offline compilers and other associated components.

In an illustrative implementation, an exemplary DNN environment cancomprise one or more processing blocks (e.g., computer processingunits—CPUs), a memory controller, a high bandwidth fabric (e.g., databus passing data and/or data elements between an exemplary DNN moduleand the cooperating components of a DNN environment), operationcontroller, and a DNN module. In the illustrative implementation, theexemplary DNN module can comprise an exemplary DNN state controller, adescriptor list controller (DLC), dMA (DDMA), DMA Streaming Activations(DSA), operation controller, load controller, and store controller.

In an illustrative operation, the operational controller of the NN/DNNenvironment can operatively process large amounts of data in order toapply one or more desired data processing operations (e.g., convolution,max pooling, scalar multiply/add, summation, fully connected, etc.). Aparticipating user can specify the dimensions of the data beingprocessed by the NN/DNN environment. Then, using the specified datadimensions, as well as the number of available processing layers in theNN/DNN environment, and data representative of one or morecharacteristics of cooperating memory components of the NN/DNNenvironment (e.g., memory size, location, latency, efficiency, etc.) aswell as characteristics about the processing units of the NN/DNNenvironment, a data apportionment sequence can be calculated by theNN/DNN environment components that specifies that input data for eachlayer is to be apportioned (as well as any associated processingparameters) and communicated between the cooperating NN/DNN memorycomponents and NN/DNN processors to achieve optimal processing.Illustratively, the data apportionment sequence can include a “breadth”only processing sequence, a “depth first” processing sequence, and/or a“dynamic depth first” processing sequence.

Illustratively, a “breadth” only processing sequence describes aprocessing sequence wherein the partitions of each layer are processedsequentially from a first processing layer to a subsequent processinglayer. A “depth first” processing sequence describes a processingsequence wherein a partition from each available processing layer isprocessed in a preferred sequence. A “dynamic depth” first processingsequence describes a processing sequence wherein the layer data isprocessed according to a combination of “breadth” only and/or “depthfirst” processing depending on various NN/DNN characteristics as well asdata characteristics. The “dynamic depth” processing sequence canoperatively jump between “breadth” only and “depth first” processing asenvironment and/or data characteristics change.

In an illustrative implementation, the exemplary NN/DD environment caninclude a local memory component and an external memory component. Thelocal memory components operatively transfer data at higher rates withreduced latency relative to the external memory component. The localmemory component can include a size to store smaller amounts of datarelative to the external memory component.

In an illustrative operation, input data (e.g., a data blob) can bereceived for processing by the NN/DNN environment having a specificdefined data dimension, associated data processing parameters (e.g.,layer weights), defined number of processing layers required forprocessing, as well as data representative of one or morecharacteristics of the cooperating memory components of the NN/DNNenvironment. Operatively, the exemplary operational controller or othercooperating NN/DNN component (e.g., online/offline compiler, etc.)provides instructions to calculate a data apportionment sequence using,the received data dimensions, cooperating memory characteristics, andnumber of layers. In the illustrative implementation, the calculateddata apportionment sequence generates data representative of the numberof portions to parcel the input data across each of the availableprocessing layers and specify the timing of loading of the portions ofdata (and their associated processing parameters) from an exemplaryexternal memory component to an internal memory component to anavailable processing unit of the NN/DNN environment.

Additionally, the calculated data apportionment sequence includesinstructions for the operational controller to communicate the dataportion(s) from the exemplary external memory to the local memory to theavailable processing unit(s) according to a calculated sequence.Illustratively, the exemplary local memory component can be utilized inthe processing sequence to store output data for processed portions ofdata for each of the layers. In the illustrative operation, uponprocessing all of the portions of a given processing layer and storingsuch output data in a local memory, the cooperating processing units canassemble the generated output portion data for the given processinglayer (stored in the exemplary local memory) to generate complete outputdata for the given processing layer. In an illustrative operation, thegenerated complete layer output data can then be stored in the exemplaryexternal memory component that results in making more of the exemplarylocal memory component's memory available for subsequent processing.

It should be appreciated that, although described in relation to asystem, the above-described subject matter may also be implemented as acomputer-controlled apparatus, a computer process, a computing system,or as an article of manufacture such as a computer-readable mediumand/or dedicated chipset.

Neural Networks Background:

In artificial neural networks, a neuron is the base unit used to model abiological neuron in the brain. The model of an artificial neuron caninclude the inner product of an input vector with a weight vector addedto a bias, with a non-linearity applied. Comparatively, a neuron, in anexemplary DNN module, (e.g., 105 of FIG. 1) is closely mapped to anartificial neuron.

Illustratively, the DNN module can be considered a superscalarprocessor. Operatively, it can dispatch one or more instructions tomultiple execution units called neurons. The execution units can be“simultaneous dispatch simultaneous complete” where each execution unitis synchronized with all of the others. A DNN module can be classifiedas a SIMD (single instruction stream, multiple data stream)architecture.

Turning to exemplary DNN environment 100 of FIG. 1, DNN module 105 has amemory subsystem with a unique L1 and L2 caching structure. These arenot traditional caches, but are designed specifically for neuralprocessing. For convenience, these caching structures have adopted namesthat reflect their intended purpose. By way of example, the L2 cache125(A) can illustratively maintain a one megabyte (1 MB) storagecapacity with a high speed private interface operating at 16 GBps. TheL1 cache can maintain an 128 KB storage capacity split between kerneland activation data. The L1 cache can be referred to as Line Buffer, andthe L2 cache is referred to as BaSRAM.

The DNN module can be a recall-only neural network and programmaticallysupport a wide variety of network structures. Training for the networkcan be performed offline in a server farm or data center; the DNN moduledoes not perform any training functions. The result of training is a setof parameters that can be known as either weights or kernels. Theseparameters represent a transform function that can be applied to aninput with the result being a classification or semantically labeledoutput.

In an illustrative operation, the DNN module can accept planar data asinput. Input is not limited to image data only, as long as the datapresented is in a uniform planar format the DNN can operate on it.

The DNN module operates on a list of layer descriptors which correspondto the layers of a neural network. Illustratively, the list of layerdescriptors can be treated by the DNN module as instructions. Thesedescriptors can be pre-fetched from memory into the DNN module andexecuted in order.

Generally, there can be two main classes of layer descriptors: 1)Memory-to-memory move descriptors, and 2) Operation descriptors.Memory-to-memory move descriptors can be used to move data to/from themain memory to/from a local cache for consumption by the operationdescriptors. Memory-to-memory move descriptors follow a differentexecution pipeline than the operation descriptors. The target pipelinefor memory-to-memory move descriptors can be the internal DMA engine,whereas the target pipeline for the operation descriptors can be theneuron processing elements. Operation descriptors are capable of manydifferent layer operations.

The output of the DNN is also a blob of data. The output can optionallybe streamed to a local cache or streamed to main memory. The DNN modulecan pre-fetch data as far ahead as the software will allow. Software cancontrol pre-fetching by using fencing and setting dependencies betweendescriptors. Descriptors that have dependency sets are prevented frommaking forward progress until the dependency has been satisfied.

Turning now to FIG. 1, an exemplary neural network environment 100 cancomprise various cooperating components inclusive of DNN module 105,cache memory 125 and 125(A), low bandwidth fabric 110, bridge component115, high bandwidth fabric 120, SOC 130, PCIE “End Point” 135, TensilicaNode 140, memory controller 145, LPDDR4 memory 105, and an input datasource 102. Further, as is shown, DNN module 105 can also comprise anumber of components comprising prefetch 105(A), DMA 105(B), RegisterInterface 105(D), load/store unit 105(C), layer controller 105(G),save/restore component 105(E), and neurons 105(F). Operatively, anexemplary DNN environment 100 can process data according to a selectedspecification wherein the DNN module performs one or more functions asdescribed herein.

FIG. 2 illustrates an exemplary neural network environment 200 operableto calculate and execute an exemplary data apportionment sequence inaccordance with the herein described systems and methods. As is shown,the exemplary neural network environment 200 comprises one or moreoperation controllers 225 that provide one or more commands forexecution. The one or more operation controllers 225 can operate togenerate instructions (e.g., data apportionment sequence 230) that arecommunicated through fabric 215 to cooperating memory component localmemory 210 as well as one or more processing units 205 (e.g., neuronprocessors). Further, as is shown in FIG. 2, data can be operativelyretrieved and stored in main memory component 220 by processing units205 through the fabric 215. In some embodiments, data can be stored fromprocessing units 205 to local memory 210 through a local fabric 235. Inan illustrative implementation, a neural network environment fabric canbe a data bus capable of accepting and communicating through variousdata to one or more cooperating components.

In the illustrative operation, the exemplary neural network environment200 can operatively process data according to the process described inFIGS. 6 and 7. Specific to the components described in FIG. 2, thesecomponents are merely illustrative, as one of ordinary skill in the artwould appreciate the processing described in FIGS. 6 and 7 to beperformed by other components than those illustrated in FIG. 2.

FIG. 3 illustrates an example logical data mapping 300 for exemplaryinput data. As is shown, data 305 can be represented as data having acertain dimension and volume 340 comprising channel count 310, height315, and width 320. According to the herein described systems andmethods, data 305 can be portioned and prepared for processing bycooperating n neurons 330 such that first portion can be communicated toa first neuron, second portion b can be communicated to a second neuron,and so forth until n portions are communicated to n neurons.

In an illustrative operation, the portions of data 305 can be determinedusing n sliding window/kernels 325 based on one or more instructionsprovided by a cooperating controller component of an exemplary neuralnetwork environment (e.g., 200 of FIG. 2). Further as is shown, theinput data portions a, b, c, and d can be addressed to a physical memory325 using one or more initialization parameters provided by acooperating operation controller component of an exemplary neuralnetwork environment (e.g., 200 of FIG. 2).

FIG. 4 illustrates an exemplary data state model and exemplary dataprocessing sequence of exemplary neural network environment 400. As isshown, neural network environment 400 can include a main memorycomponent 220, local memory component 210, and processing units 205 (asare shown in FIG. 2). Operatively, data can be directly loaded from mainmemory 220 to local memory 210 such that an exemplary first layer inputdata layer 1 input 410 is loaded from main memory 220 to local memory210. Additionally, the layer 1 weights 420 (e.g., processing parametersassociated with layer 1 input 410) can also be loaded from main memory220 to local memory 210. As is shown in FIG. 1, the layer 1 input andlayer 1 weights can then be communicated to processing units 205 forprocessing by process layer 1 430. Operatively, processing units 205 canprocess layer 1 input data 410 and layer 1 weights 420 to generateoutput data (not shown) that then can be stored as layer 1 outputs 440to main memory 220. This sequence comprises the complete layer oneprocessing.

Further, as is shown in FIG. 4, layer 2 input data 450 and itsassociated layer 2weights 460 can be loaded from main memory 220 tolocal memory 210. Operatively, layer 2 input data 450 and layer 2weights 460 can then be communicated from local memory 210 to processingunits 205 for processing. In an illustrative operation, processing unit205 can process layer 2 input data 450 and layer 2 weights 460 to atstepwise processing sequence step 470 to generate layer 2 output data480 for storage on main memory 220.

In an illustrative operation, the generated layer 1 output data store onmain memory by sequence step 440 can be layer 2 input data 450. Suchsequential layer processing wherein the output of the previousprocessing layer can act as input to a subsequent processing layer istypical of conventional deep neural network environment processingprotocols. The exemplary data processing sequence of FIG. 4 relies oneach layer input data being fully processed prior to the subsequentlayer input data being processed by the processing units 205 of theexemplary neural network environment 400.

For exemplary purposes only, FIG. 4 depicts exemplary neural networkenvironment 400 including two processing layers having main memorycomponent 220, local memory component 210, and processing units 205 in aparticular cooperating configuration. One of ordinary skill willappreciate that such depiction is merely illustrative as the inventiveconcepts described by the herein described systems and methodscontemplate operations for neural network environments having variousprocessing layers and supporting various cooperating configurationsbetween the main memory 220 component, local memory component, andprocessing units 205.

FIG. 5A illustrates another exemplary data stepwise sequence processingmodel of exemplary network environment 500. As is shown, exemplarynetwork environment 500 can include main memory component 220, localmemory component 210, and processing units 205 (as are shown in FIG. 2).By way of illustration only, the exemplary neural network environment isdepicted as supporting two processing layers. In this illustration, oneor more neural network environment 500 components (not shown) (e.g.,operational controller(s) 225) can calculate a data apportionmentsequence for the completed input data of layer 1 (not shown). The dataapportionment sequence can be calculated using various characteristicsof layer 1 input data (e.g., data dimensions), as well as variouscharacteristics of the main memory and local memory components (e.g.,memory size, memory latency, memory location, etc.), and the number ofprocessing layers available in the neural network environment (e.g., inthis illustration—two layers).

Operatively, the calculated apportionment sequence can includeinstructions executable by one or more components of exemplary neuralnetwork environment 500 (e.g., DNN module 105 of FIG. 1) to portion theinput data for each of the layers as well as the associated processingparameters for each input data layer portion into optimal data portionsizes that when processed by exemplary neural network environment 500would optimize the utility of the one or more cooperating memorycomponents (e.g., 220, 210) during a processing cycle. Additionally, inan illustrative operation, the calculated apportionment sequence caninclude a stepwise processing sequence in which the apportioned inputlayer data is communicated between/processed by, in discrete steps, thecooperating memory components (220 and 210) as well as processing units205. Furthermore, the exemplary calculated apportionment sequence caninclude instructions to store intermediate output layer data in localmemory according to the provided stepwise processing sequence.

FIG. 5A shows the processing of input data in an exemplary neuralnetwork environment having two processing layers that supports a localmemory component 210, main memory component 220, and one or moreprocessing units 205. Illustratively, the input data can be apportionedaccording to an exemplary calculated data apportionment sequence toinclude a layer 1 part 1 portion (and associated layer 1 weights), layer1 part 2 portion, and layer 2 weights. Additionally, in the illustrativeimplementation, an exemplary calculated apportionment sequence caninclude instructions regarding when such portions of data that have beenapportioned are loaded between the available memory components (localmemory 210 and main memory 220) and the processing units 205 (e.g.,exemplary stepwise processing sequence). Intermediate layer output datastorage in local memory 210 according the exemplary stepwise processingsequence can also be managed by one or more instructions found in thecalculated apportionment sequence.

According to an exemplary calculated apportionment sequence (not shown)layer 1 input part 1 (i.e., apportioned according to the exemplaryapportionment sequence) can be loaded from main memory component 220 tolocal memory component 210 at stepwise processing sequence step 505.Additionally, layer 1 weights (e.g., processing parameters for layer 1part 1 input data 505), can be loaded from main memory component 220 tolocal memory component 220 at stepwise processing sequence step 510.Processing units 205 can process layer 1 part 1 input data and layer 1weights according to stepwise processing sequence step 515 to generateand store layer 1 part 1 output data at stepwise processing sequencestep 520. Processing proceeds along the stepwise processing sequence tostep 525 where layer 2 weights are loaded from main memory 220 to localmemory 210 to allow for the processing of layer 2, part 1 input data(i.e., as described herein, the output of a given processing layer canact as the input of a subsequent layer in which, in this illustrativeimplementation, the output of layer 1 is the input data for layer 2) byprocessing units 205 at stepwise processing sequence step 530. The layer2, part 1 output data can then be stored in main memory 220 at stepwiseprocessing sequence step 535.

Operatively, the remaining portion of layer 1 input data (i.e., layer 1part 2 input data) is then loaded from main memory 220 to local memory210 at stepwise processing sequence step 540 and processed at stepwiseprocessing sequence step 545 to generate layer 1 part 2 output datawhich is then stored in local memory 210 at stepwise processing step550. The remaining input data portion of layer 2 (i.e., layer 2 portion2 is also layer 1 part 2 output data) is then loaded from local memory210 for processing by processing units 205 at stepwise processingsequence step 555. Layer 2 part 2 output data is generated for storageinto main memory 220 at stepwise processing sequence step 560 andcompletes the generation of the complete output data set by exemplaryneural network environment 500 for the originally received input data ofthe illustrative implementation (i.e., since there are no moreprocessing layers, layer 2 part 1 output data and layer 2 part 2 outputdata store in main memory can represent that totality of the output datato be generated by exemplary neural network environment 500 for theoriginally received input data).

For exemplary purposes, only, FIG. 5A depicts exemplary neural networkenvironment 500 including two processing layers having main memorycomponent 220, local memory component 210, and processing units 205 in aparticular cooperating configuration. One of ordinary skill willappreciate that this depiction is merely illustrative as the inventiveconcepts described by the herein described systems and methodscontemplate operations for neural network environments having variousprocessing layers and supporting various cooperating configurationsbetween the main memory 220 component, local memory component 210, andprocessing units 205 as well as various stepwise processing sequencesthat can differ based on the characteristics of the components ofexemplary neural network environment 500.

FIG. 5B is a block diagram showing exemplary processing sequences 570,575, and 580 of an exemplary neural network environment 500A. As isshown, processing sequence 570 can to process partitions (570(a),570(b), 570(c), 570(d), 570(e), 570(f), 570(g), 570(h), 570(i), 570(j),570(k), and 570(k)—for the sake of simplicity to be referred tohereafter as 570(a) - 570(1))) of data from available processing layers(Layer 0, Layer 1, and Layer 2). In this processing sequence, the datapartitions (570(a)-570 (l)) are processed in sequential order (asindicated by the processing sequence steps 1-12) (e.g., Layer 0; 1-4,Layer 1; 5-8, and Layer 2;9-12) such that the data partitions of Layer 0(i.e., 570(a), 570(b), 570(c), and 570(d) are first processed, then thedata partitions of Layer 1 (i.e., 570(e), 570(f), 570(g), and 570(h)),then the data partitions of Layer 2 (i.e., 570(i), 570(j), 570(k), and570(l)). This type of processing sequence can be considered a “breadth”only processing sequence.

Further, as is shown in FIG. 5B, exemplary NN/DNN environment can alsoexecute processing sequence 575. In this processing sequence, a datapartition from each layer is processed in sequence (i.e., 1—(575(a),2—575(e), 3—575(i)), then a second data partition from each layer isprocessed in sequence (i.e., 4—(575(b), 5—575(f), 6—575(j)) until alldata partitions are processed across the layers. This type of processingsequence can be considered a “depth first” processing sequence.

Further as is shown in FIG. 5B, exemplary NN/DNN environment can alsoexecute processing sequence 580. In this processing sequence, datapartitions (580(a), 580(b), 580(c), 580(d), 580(e), 580(f), 580(g),580(h), 580(i), 580(j), 580(k), and 580(l)) can be processed accordingto “breadth” only and/or “depth first” processing sequence such that, inan illustrative operation, data partitions from each layer can beprocessed sequentially (e.g., 1—580(a), 2—580(e), 3—(580(i)) accordingto a “depth first” processing sequence, and then data from a singlepartition can be processed (4—580(b), 5—580(c), then 6—580(f), 7—580(g),and then 8—(580(j), 9—580(k)) according to a “breadth” only processingsequence, and then data from each partition can be processed (10—580(d),11—580(h), 12—580(l)) according to a “depth first” processing sequence.Accordingly, the “dynamic depth first” processing sequence canoperatively allow for the processing of the layer partitions through anexemplary interleaving operation to switch from a “breadth” only to a“depth first” processing sequence as the data and environmentcharacteristics dictate. The “dynamic depth first” processing sequencecan provide additional robustness to optimize performance and memoryusage depending on the environment and/or data characteristics.

It is appreciated that the exemplary processing sequences illustrated inFIG. 5B are merely illustrative, as the inventive concepts describedherein can utilize other processing sequences that operatively optimizeperformance and/or memory utilization for exemplary NN/DNN environments.Additionally, although the size of the partitions are shown as being ofequal size, the inventive concepts described herein contemplate datapartitions of varying sizes.

FIG. 6 is a flow diagram of an illustrative process 600 to enhanceperformance of a neural network environment using a depth first approachto data processing. It should be understood that the operations of theprocesses disclosed herein are not necessarily presented in anyparticular order and that performance of some or all of the operationsin an alternative order(s) is possible and is contemplated. Theoperations have been presented in the demonstrated order for ease ofdescription and illustration. Operations may be added, omitted, and/orperformed simultaneously, without departing from the scope of theappended claims.

It also should be understood that the illustrated processes (alsoreferred to as “methods” or “routines”) can end at any time and need notbe performed in its entirety. Some or all operations of the methods,and/or substantially equivalent operations, can be performed byexecution of computer-readable instructions included on acomputer-storage media, as defined below. The term “computer-readableinstructions,” and variants thereof, as used in the description andclaims, is used expansively herein to include routines, applications,application modules, program modules, programs, components, datastructures, algorithms, and the like. Computer-readable instructions canbe implemented on various system configurations, includingsingle-processor or multiprocessor systems, minicomputers, mainframecomputers, personal computers, hand-held computing devices,microprocessor-based, programmable consumer electronics, combinationsthereof, and the like.

Thus, it should be appreciated that the logical operations describedherein are implemented (1) as a sequence of computer implemented acts orprogram modules running on a computing system and/or (2) asinterconnected machine logic circuits or circuit modules within thecomputing system. The implementation is a matter of choice dependent onthe performance and other requirements of the computing system.Accordingly, the logical operations described herein are referred tovariously as states, operations, structural devices, acts, or modules.These operations, structural devices, acts, and modules may beimplemented in software, in firmware, in special purpose digital logic,and any combination thereof.

For example, the operations of the process 600 are described herein asbeing implemented, at least in part, by the components describe hereinand/or components of a remote system. In some configurations, thecomponents described herein or another module running the featuresdisclosed herein can be a dynamically linked library (DLL), a staticallylinked library, functionality produced by an application programinginterface (API), a compiled program, an interpreted program, microcode,machine code, a script or any other executable set of instructions. Datacan be stored in a data structure in one or more memory components. Datacan be retrieved from the data structure by addressing links orreferences to the data structure.

Although the following illustration refers to the components of thefigures, it can be appreciated that the operations of the routines maybe also implemented in many other ways. For example, the process 600 maybe implemented, at least in part, by a processor of another remotecircuit or a local circuit. In addition, one or more of the operationsof the process 600 may alternatively or additionally be implemented, atleast in part, by a chipset working alone or in conjunction with othersoftware modules. Any service, circuit or application suitable forproviding the techniques disclosed herein can be used in operationsdescribed herein.

As is shown, processing begins at block 605 where one or moreinitialization parameters are received from one or more cooperatingcomponents of a neural networking computing environment. Processing thenproceeds to block 610 where one more data processing commands arereceived from one or more cooperating components of a neural networkingcomputing environment. At block 615, an input data apportionmentsequence can be calculated that provides a stepwise processing sequenceand associated instructions for portioning, processing, and storing oneor more of the apportioned data (e.g., input or generated output data)for each layer in one or more cooperating memory components of anexemplary neural network environment according to one or more of“breadth” only, “depth first” and “dynamic depth first” processingsequence. Processing then proceeds to block 620 where a portion of theapportioned input data or generated intermediate output data isprocessed according to a stepwise processing sequence of the calculateddata apportionment sequence of block 615. Processing then proceeds toblock 625 where the output of the processed apportioned data can bestored in one or more of the cooperating memory components of theexemplary neural network environment (e.g., in a local or external/mainmemory component) according to the calculated data apportionmentsequence. A check is then performed at block 630 to determine if all ofthe steps of the stepwise processing sequence have been completed.

If the check at block 630 indicates that there are no additional stepsto be processed of the stepwise processing sequence, processingterminates at block 635. However, if the check at block 630 indicatesthat there are additional steps to be processed of the stepwiseprocessing sequence (e.g., there are more layer portions of data to beprocessed), processing reverts back to block 620 and proceeds fromthere.

FIG. 7 is a flow diagram of an illustrative process 700 that utilizesdepth first processing to enhance overall processing performance of anexemplary neural network environment. As is shown, processing begins atblock 705 where one or more initialization parameters are received froma cooperating component of the neural network environment (e.g.,operation controller) wherein the one or more initialization parameterscan include data representative of the dimensions for input data, one ormore characteristics of one or more memory components of the exemplaryneural network environment, and/or the number of processing layers ofthe exemplary neural network environment. Processing then proceeds toblock 710 where an input data apportionment sequence can be calculatedusing input data dimension data that provides a stepwise processingsequence and associated instructions for portioning, processing, andstoring one or more of the apportioned data (e.g., input or generatedoutput data) for each layer in one or more cooperating memory componentsof an exemplary neural network environment. Processing then proceeds toblock 715 where one or more processing instructions are received by oneor more cooperating components of the exemplary neural networkenvironment (e.g., processing units/neural processors) to process aspecified portion of the apportioned data according to an exemplarystepwise processing sequence (e.g., according to one or more of“breadth” only, “depth first” and “dynamic depth first” processingsequence). Processing then proceeds to block 720 where a specifiedportion of the apportioned data is processed according to a stepwiseprocessing sequence using the received one or more processinginstructions. At block 725, the output of the processed apportioned datais then stored in one or more cooperating memory components (e.g., localmemory or external/main memory) according to the exemplary stepwiseprocessing sequence.

A check is then performed at block 730 to determine if the all of thesteps of the stepwise processing sequence have been completed. If thecheck at block 730 indicates that there are no additional steps to beprocessed of the stepwise processing sequence, processing terminates atblock 735. However, if the check at block 730 indicates that there areadditional steps to be processed of the stepwise processing sequence(e.g., there are more layer portions of data to be processed),processing reverts back to block 715 and proceeds from there.

The computer architecture 800 illustrated in FIG. 8 includes a centralprocessing unit 802 (“CPU”), a system memory 804, including arandom-access memory 806 (“RAM”) and a read-only memory (“ROM”) 808, anda system bus 810 that couples the memory 804 to the CPU 802. A basicinput/output system containing the basic routines that help to transferinformation between elements within the computer architecture 800, suchas during startup, is stored in the ROM 808. The computer architecture800 further includes a mass storage device 812 for storing an operatingsystem 814, other data, and one or more application programs.

The mass storage device 812 is connected to the CPU 802 through a massstorage controller (not shown) connected to the bus 810. The massstorage device 812 and its associated computer-readable media providenon-volatile storage for the computer architecture 800. Although thedescription of computer-readable media contained herein refers to a massstorage device, such as a solid-state drive, a hard disk or CD-ROMdrive, it should be appreciated by those skilled in the art thatcomputer-readable media can be any available computer storage media orcommunication media that can be accessed by the computer architecture800.

Communication media includes computer readable instructions, datastructures, program modules, or other data in a modulated data signalsuch as a carrier wave or other transport mechanism and includes anydelivery media. The term “modulated data signal” means a signal that hasone or more of its characteristics changed or set in a manner as toencode information in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of the any of the aboveshould also be included within the scope of computer-readable media.

By way of example, and not limitation, computer storage media mayinclude volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer-readable instructions, data structures, program modules orother data. For example, computer media includes, but is not limited to,RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memorytechnology, CD-ROM, digital versatile disks (“DVD”), HD-DVD, BLU-RAY, orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed bythe computer architecture 800. For purposes of the claims, the phrase“computer storage medium,” “computer-readable storage medium” andvariations thereof, does not include waves, signals, and/or othertransitory and/or intangible communication media, per se.

According to various techniques, the computer architecture 800 mayoperate in a networked environment using logical connections to remotecomputers through a network 820 and/or another network (not shown). Thecomputer architecture 800 may connect to the network 820 through anetwork interface unit 816 connected to the bus 810. It should beappreciated that the network interface unit 816 also may be utilized toconnect to other types of networks and remote computer systems. Thecomputer architecture 800 also may include an input/output controller818 for receiving and processing input from a number of other devices,including a keyboard, mouse, or electronic stylus (not shown in FIG. 8).Similarly, the input/output controller 818 may provide output to adisplay screen, a printer, or other type of output device (also notshown in FIG. 8). It should also be appreciated that via a connection tothe network 820 through a network interface unit 816, the computingarchitecture may enable DNN module 105 to communicate with the computingenvironments 100.

It should be appreciated that the software components described hereinmay, when loaded into the CPU 802 and/or the DNN Module 105 andexecuted, transform the CPU 802 and/or the DNN Module 105 and theoverall computer architecture 800 from a general-purpose computingsystem into a special-purpose computing system customized to facilitatethe functionality presented herein. The CPU 802 and/or the DNN Module105 may be constructed from any number of transistors or other discretecircuit elements and/or chipsets, which may individually or collectivelyassume any number of states. More specifically, the CPU 802 and/or theDNN Module 105 may operate as a finite-state machine, in response toexecutable instructions contained within the software modules disclosedherein. These computer-executable instructions may transform the CPU 802by specifying how the CPU 802 transitions between states, therebytransforming the transistors or other discrete hardware elementsconstituting the CPU 802.

Encoding the software modules presented herein also may transform thephysical structure of the computer-readable media presented herein. Thespecific transformation of physical structure may depend on variousfactors, in different implementations of this description. Examples ofsuch factors may include, but are not limited to, the technology used toimplement the computer-readable media, whether the computer-readablemedia is characterized as primary or secondary storage, and the like.For example, if the computer-readable media is implemented assemiconductor-based memory, the software disclosed herein may be encodedon the computer-readable media by transforming the physical state of thesemiconductor memory. For example, the software may transform the stateof transistors, capacitors, or other discrete circuit elementsconstituting the semiconductor memory. The software also may transformthe physical state of such components in order to store data thereupon.

As another example, the computer-readable media disclosed herein may beimplemented using magnetic or optical technology. In suchimplementations, the software presented herein may transform thephysical state of magnetic or optical media, when the software isencoded therein. These transformations may include altering the magneticcharacteristics of particular locations within given magnetic media.These transformations also may include altering the physical features orcharacteristics of particular locations within given optical media, tochange the optical characteristics of those locations. Othertransformations of physical media are possible without departing fromthe scope and spirit of the present description, with the foregoingexamples provided only to facilitate this discussion.

In light of the above, it should be appreciated that many types ofphysical transformations take place in the computer architecture 800 inorder to store and execute the software components presented herein. Italso should be appreciated that the computer architecture 800 mayinclude other types of computing devices, including hand-held computers,embedded computer systems, personal digital assistants, and other typesof computing devices known to those skilled in the art. It is alsocontemplated that the computer architecture 800 may not include all ofthe components shown in FIG. 8, may include other components that arenot explicitly shown in FIG. 8, or may utilize an architecturecompletely different than that shown in FIG. 8.

Computing system 800, described above, can be deployed as part of acomputer network. In general, the above description for computingenvironments applies to both server computers and client computersdeployed in a network environment.

FIG. 9 illustrates an exemplary networked computing environment 900,with a server in communication with client computers via acommunications network, in which the herein described apparatus andmethods may be employed. As shown in FIG. 9, server(s) 905 may beinterconnected via a communications network 820 (which may be either of,or a combination of, a fixed-wire or wireless LAN, WAN, intranet,extranet, peer-to-peer network, virtual private network, the Internet,Bluetooth communications network, proprietary low voltage communicationsnetwork, or other communications network) with a number of clientcomputing environments such as a tablet personal computer 910, a mobiletelephone 915, a telephone 920, a personal computer(s)801, a personaldigital assistant 925, a smart phone watch/personal goal tracker (e.g.,Apple Watch, Samsung, FitBit, etc.) 930, and a smart phone 935. In anetwork environment in which the communications network 820 is theInternet, for example, server(s) 905 can be dedicated computingenvironment servers operable to process and communicate data to and fromclient computing environments 801, 910, 915, 920, 925, 930, and 935 viaany of a number of known protocols, such as, hypertext transfer protocol(HTTP), file transfer protocol (FTP), simple object access protocol(SOAP), or wireless application protocol (WAP). Additionally, thenetworked computing environment 900 can utilize various data securityprotocols such as secured socket layer (SSL) or pretty good privacy(PGP). Each of the client computing environments 801, 910, 915, 920,925, 930, and 935 can be equipped with computing environment 905operable to support one or more computing applications or terminalsessions such as a web browser (not shown), or other graphical userinterface (not shown), or a mobile desktop environment (not shown) togain access to the server computing environment(s) 905.

Server(s) 905 may be communicatively coupled to other computingenvironments (not shown) and receive data regarding the participatinguser's interactions/resource network. In an illustrative operation, auser (not shown) may interact with a computing application running on aclient computing environment(s) to obtain desired data and/or computingapplications. The data and/or computing applications may be stored onserver computing environment(s) 905 and communicated to cooperatingusers through client computing environments 905, 910, 915, 920, 925,930, and 935, over an exemplary communications network 820. Aparticipating user (not shown) may request access to specific data andapplications housed in whole or in part on server computingenvironment(s) 905. These data may be communicated between clientcomputing environments 801, 910, 915, 920, 925, 930, 935 and servercomputing environment(s) 905 for processing and storage. Servercomputing environment(s) 905 may host computing applications, processesand applets for the generation, authentication, encryption, andcommunication of data and applications and may cooperate with otherserver computing environments (not shown), third party service providers(not shown), network attached storage (NAS) and storage area networks(SAN) to realize application/data transactions.

Example Clauses

The disclosure presented herein may be considered in view of thefollowing clauses.

Example Clause A, a system for enhanced data processing in a neuralnetwork environment, the system comprising at least one processor, atleast one first memory component, and at least one second memorycomponent in communication with the at least one processor, the at leastone first and/or second memory components having computer-readableinstructions stored thereupon that, when executed by the at least oneprocessor, cause the at least one processor to receive one or moreinitialization parameters from a cooperating controller component of theneural network environment, the initialization parameters comprisingdata representative of the dimensions of the data to be processed by theneural network environment, calculate an apportionment sequence for thedata, the apportionment sequence comprising one or more instructions tostore output data in the at least one first memory component and/or theat least second memory component), the apportionment sequence comprisingbreadth only, depth first, and dynamic depth first processing sequence,load data from a cooperating memory component of the neural networkenvironment, receive one or more instructions from the cooperatingcontroller component of the neural network environment to process aselected portion of the data according to the calculated apportionmentsequence, process the portion of data by one or more cooperatingprocessing units to generate output data for storage on the at least onefirst memory component or the at least one second memory component, andstore the generated output data on the at least one first memorycomponent or the at least one second memory components.

Example Clause B, the system of Example Clause A, wherein the calculatedapportionment sequence is based on the number of layers of the neuralnetwork environment.

Example Clause C, the system of Example Clause A and B, wherein thecomputer-readable instructions further cause the at least one processorto load data representative of one or more layer weights for use by oneor more processing units (205) of the neural network environment togenerate the output data.

Example Clause D, the system of Example Clauses A through C, wherein thecomputer-readable instructions further cause the at least one processorto store the generated output data in the at least one first memorycomponent.

Example Clause E, the system of Example Clauses A through D, wherein thecomputer-readable instructions further cause the at least one processorto store the generated output data in the at least one second memorycomponent.

Example Clause F, the system of Example Clauses A through E, wherein thecomputer-readable instructions further cause the at least one processorto calculates the apportionment sequence based on the size of the atleast first memory component and/or the size of the at least secondmemory component.

Example Clause G, the system of Example Clauses A through F, wherein thecomputer-readable instructions further cause the at least one processorto store the generated output data in the at least first memorycomponent.

Example Clause H, the system of Example Clauses A through G, wherein thecomputer-readable instructions further cause the at least one processorto store the generate output data in the at least second memorycomponent.

Example Clause I, a computer-implemented method, comprising: receivingone or more initialization parameters from a cooperating controllercomponent, the initialization parameters comprising data representativeof the dimensions of input data and data representative of the number ofprocessing layers associated with the input data, calculating anapportionment sequence for the input data, the apportionment sequencecomprising one or more instructions to store output data in at least onefirst memory component and/or at least one second memory component andcomprising one or more instructions to load associated data required toprocess the input data), the apportionment sequence comprising breadthonly, depth first, and dynamic depth first processing sequence, loaddata representative of one or more processing weights from a cooperatingmemory component, receiving one or more instructions from thecooperating controller component to process a selected portion of thedata according to the calculated apportionment sequence, processing theportion of data according to the calculated apportionment sequence byone or more cooperating processing units to generate output data forstorage on the at least one first memory component or the at least onesecond memory component, and storing the generated output data on the atleast one first memory component or the at least one second memorycomponents.

Example Clause J, the computer-implemented method of Clause I, furthercomprising receiving data representative of the size of the at least onefirst memory component and the size of the at least one second memorycomponent for use in calculating the apportionment sequence.

Example Clause K, the computer-implemented method of Clauses I and J,further comprising storing the generated output data on the at leastfirst memory component.

Example Clause L, the computer-implemented method of Clauses I throughK, further comprising storing the generated output data on the at leastone second memory component.

Example Clause M, the computer-implemented method of Clauses I throughL, further comprising storing a portion of the generated output data onthe at least first memory component.

Example Clause N, the computer-implemented method of Clauses I throughM, further comprising loading processing weight data associated to theportion of data being processed along the calculated apportionmentsequence.

Example Clause O, the computer-implemented method of Clauses I throughN, further comprising loading processing weight data associated for theentirety of the input data for each processing layer.

Example Clause P, the computer-implemented method of Clauses I throughO, further comprising processing the output data of the previousprocessing layer as input data to a subsequent processing layer.

Example Clause Q, a computer-readable storage medium havingcomputer-executable instructions stored thereupon which, when executedby one or more processors of a computing device, cause the one or moreprocessors of the computing device to: receive one or moreinitialization parameters from a cooperating controller component of theneural network environment, the initialization parameters comprisingdata representative of the dimensions of the data to be processed by theneural network environment, calculate an apportionment sequence for thedata, the apportionment sequence comprising one or more instructions tostore output data in the at least one first memory component and/or theat least second memory component and load data comprising input data andassociated input data parameters between the at least one first memorycomponent and the at least one second memory component), theapportionment sequence comprising breadth only, depth first, and dynamicdepth first processing sequence, load data from a cooperating memorycomponent of the neural network environment, receive one or moreinstructions from the cooperating controller component of the neuralnetwork environment to process a selected portion of the data accordingto the calculated apportionment sequence, process the portion of data byone or more cooperating processing units using the associated input dataparameters to generate output data for storage on the at least one firstmemory component or the at least one second memory component; and storethe generated output data on the at least one first memory component orthe at least one second memory components according to the calculatedapportionment sequence.

Example Clause R, the computer-readable storage medium of Clause Q,wherein the instructions further cause the one or more processors of thecomputing device to: load data representative of processing weights forthe portion of data from an external memory component to a local memorycomponent of the neural network environment according to the calculatedapportionment sequence.

Example Clause S, the computer-readable storage medium of Clauses Q andR, wherein the instructions further cause the one or more processors ofthe computing device to: store the generated output data for the portionof the data in the local memory component according to the calculatedapportionment sequence.

Example Clause T, the computer-readable storage medium of Clauses Qthrough S, wherein the instructions further cause the one or moreprocessors of the computing device to: generate output data for thefinal processing layer of the neural network environment.

Example Clause U, the computer-readable storage medium of Clauses Qthrough T, wherein the instructions further cause the one or moreprocessors of the computing device to: store the generated finalprocessing layer output data in the external memory component.

Example Clause V, the computer readable storage medium of Clauses Qthrough U, wherein the memory component cooperates with a physicalsensor capable of producing input data comprising audio data, videodata, hepatic sensory data, and other data for subsequent processing bythe one or more cooperating processing units\ Example Clause W, thecomputer readable storage medium of Clauses Q through V, wherein thecooperating processing units electronically cooperate with one or moreoutput physical components operative to receive for human interactionprocessed input data comprising audio data, video data, hepatic sensorydata and other data.

CONCLUSION

In closing, although the various techniques have been described inlanguage specific to structural features and/or methodological acts, itis to be understood that the subject matter defined in the appendedrepresentations is not necessarily limited to the specific features oracts described. Rather, the specific features and acts are disclosed asexample forms of implementing the claimed subject matter.

1. A system for enhanced data processing in a neural networkenvironment, the system comprising: at least one processor; at least onefirst memory component; and at least one second memory component incommunication with the at least one processor, the at least one firstand/or second memory component having computer-readable instructionsstored thereupon that, when executed by the at least one processor,cause the at least one processor to: receive one or more initializationparameters from a cooperating controller component of the neural networkenvironment, the initialization parameters comprising datarepresentative of dimensions of data to be processed by the neuralnetwork environment; calculate an apportionment sequence for the data,the apportionment sequence comprising one or more instructions to storeoutput data in the at least one first memory component and/or the atleast one second memory component, the apportionment sequence comprisingbreadth only, depth first, and dynamic depth first processing sequence;load data from the cooperating memory component of the neural networkenvironment; receive one or more instructions from the cooperatingcontroller component of the neural network environment to process aselected portion of the data according to the calculated apportionmentsequence; process the portion of data by one or more cooperatingprocessing units to generate the output data for storage on the at leastone first memory component or the at least one second memory component;and store the generated output data on the at least one first memorycomponent or the at least one second memory component.
 2. The system ofclaim 1, wherein the calculated apportionment sequence is based on thenumber of layers of the neural network environment.
 3. The system ofclaim 2, wherein the computer-readable instructions further cause the atleast one processor to load data representative of one or more layerweights for use by one or more processing units of the neural networkenvironment to generate the output data.
 4. The system of claim 3,wherein the computer-readable instructions further cause the at leastone processor to store the generated output data in the at least onefirst memory component.
 5. The system of claim 3, wherein thecomputer-readable instructions further cause the at least one processorto store the generated output data in the at least one second memorycomponent.
 6. The system of claim 1, wherein the computer-readableinstructions further cause the at least one processor to calculate theapportionment sequence based on the size of the at least one firstmemory component and/or the size of the at least one second memorycomponent.
 7. The system of claim 6, wherein the computer-readableinstructions further cause the at least one processor to store thegenerated output data in the at least one first memory component.
 8. Thesystem of claim 6, wherein the computer-readable instructions furthercause the at least one processor to store the generated output data inthe at least one second memory component.
 9. A computer-implementedmethod, comprising: receiving one or more initialization parameters froma cooperating controller component, the initialization parameterscomprising data representative of dimensions of input data and datarepresentative of the number of processing layers associated with theinput data; calculating an apportionment sequence for the input data,the apportionment sequence comprising one or more instructions to storeoutput data in at least one first memory component and/or at least onesecond memory component and comprising one or more instructions to loadassociated data required to process the input data; load datarepresentative of one or more processing weights from a cooperatingmemory component; receiving one or more instructions from thecooperating controller component to process a selected portion of thedata according to the calculated apportionment sequence), theapportionment sequence comprising breadth only, depth first, and dynamicdepth first processing sequence; processing the portion of dataaccording to the calculated apportionment sequence by one or morecooperating processing units to generate output data for storage on theat least one first memory component or the at least one second memorycomponent; and storing the generated output data on the at least onefirst memory component or the at least one second memory component. 10.The computer-implemented method of claim 9, further comprising receivingdata representative of the size of the at least one first memorycomponent and the size of the at least one second memory component foruse in calculating the apportionment sequence.
 11. Thecomputer-implemented method of claim 10, further comprising storing thegenerated output data on the at least one memory component comprisingany of a first memory component and a second memory component. 12.(canceled)
 12. The computer-implemented method of claim 9, furthercomprising storing a portion of the generated output data on the atleast one first memory component.
 13. The computer-implemented method ofclaim 9, further comprising loading processing weight data associatedwith the portion of data being processed along the calculatedapportionment sequence.
 14. The computer implemented method of claim 9,further comprising loading processing weight data associated with theentirety of the input data for each processing layer.
 15. Thecomputer-implemented method of claim 9, further comprising processingthe output data of the previous processing layer as input data to asubsequent processing layer.
 16. A computer-readable storage mediumhaving computer-executable instructions stored thereupon which, whenexecuted by one or more processors of a computing device, cause the oneor more processors of the computing device to: receive one or moreinitialization parameters from a cooperating controller component of aneural network environment, the initialization parameters comprisingdata representative of dimensions of data to be processed by the neuralnetwork environment; calculate an apportionment sequence for the data,the apportionment sequence comprising one or more instructions to storeoutput data in at least one first memory component and/or at least onesecond memory component and load data comprising input data andassociated input data parameters between the at least one first memorycomponent and the at least one second memory component), theapportionment sequence comprising breadth only, depth first, and dynamicdepth first processing sequence; load data from a cooperating memorycomponent of the neural network environment; receive one or moreinstructions from the cooperating controller component of the neuralnetwork environment to process a selected portion of the data accordingto the calculated apportionment sequence; process the portion of data byone or more cooperating processing units using the associated input dataparameters to generate output data for storage on the at least one firstmemory component or the at least one second memory component; and storethe generated output data on the at least one first memory component orthe at least one second memory component according to the calculatedapportionment sequence.
 17. The computer-readable storage medium ofclaim 16, wherein the instructions further cause the one or moreprocessors of the computing device to: load data representative ofprocessing weights for the portion of data from an external memorycomponent to a local memory component of the neural network environmentaccording to the calculated apportionment sequence.
 18. Thecomputer-readable storage medium of claim 17, wherein the instructionsfurther cause the one or more processors of the computing device to:store the generated output data for the portion of the data in the localmemory component according to the calculated apportionment sequence. 19.The computer-readable storage medium of claim 18, wherein theinstructions further cause the one or more processors of the computingdevice to: generate output data for the subsequent processing layer ofthe neural network environment.
 20. The computer-readable storage mediumof claim 19, wherein the instructions further cause the one or moreprocessors of the computing device to: store the generated finalprocessing layer output data in the external memory component.
 21. Thecomputer readable medium of claim 16, wherein the memory componentcooperates with a physical sensor capable of producing input datacomprising audio data, video data, hepatic sensory data, and other datafor subsequent processing by the one or more cooperating processingunits.
 22. The computer readable medium of claim 21, wherein thecooperating processing units electronically cooperate with one or moreoutput physical components operative to receive for human interactionprocessed input data comprising audio data, video data, hepatic sensorydata and other data.